Display device, terminal device, and display method

ABSTRACT

Movement of a display device is detected, and an image is displayed in stereoscopic display or planar display depending on the detected movement.

CROSS-REFERENCE TO RELATED PATENT APPLICATION

This application is a Continuation of U.S. application Ser. No.13/129,753, filed May 17, 2011, in the United Stated Patent andTrademark Office, which is the National Phase of PCT/JP2009/067469,filed Oct. 7, 2009, which is based upon and claims the benefit ofpriority from Japanese Patent Application No. 2008-300965 filed on Nov.26, 2008, the entire disclosure of which is incorporated herein byreference.

1. FIELD

The present invention relates to a display device and a display methodfor displaying images, and more particularly to a display device, aterminal device, and a display method for displaying stereoscopicimages.

2. DESCRIPTION OF THE RELATED ART

With the developments of mobile phones and PDAs (Personal DigitalAssistants) in recent years, efforts have been made in the art toproduce smaller-size and higher-definition display devices. Attentionhas been drawn to stereoscopic display devices as a new extra value thatcan be added to mobile devices. Generally, a means for displayingstereoscopic images relies on a process of projecting images having abinocular disparity respectively to the left and right eyes. There is astereoscopic display device including a display panel which has alenticular lens or a parallax barrier as an image swapping means.Another stereoscopic display device is of the time division type whichincludes two light sources for applying light to the right and left eyesto project left and right parallax images to the right and left eyes(see, for example, Patent document 1).

The stereoscopic display devices of the above types are suitable for useon mobile devices in that they do not require the observer to wearspecial glasses and hence to take the trouble of wearing glasses.Actually, mobile phones incorporating parallax-barrier stereoscopicdisplay devices are available as commercial products (see, for example,Non-patent document 1).

According to the above principles, however, since spatially separateparallax images are projected, the observer can see proper stereoscopicimages in a limited area. The area in which the observer can seestereoscopic images is called a stereoscopic viewing area, and isdetermined when the stereoscopic display device is designed. If thepositions of the eyes of the observer are shifted out of thestereoscopic viewing area, then problems arise in that the left imageand the right image may look overlapping (so-called dual image) and animage with a reversed protrusion depth (so-called pseudo-stereoscopicimage) may be observed.

The stereoscopic viewing area will be described below.

First, a stereoscopic viewing area achieved when a parallax barrier isused as an image swapping means will be described below.

FIG. 1 shows by way of example an optical model wherein parallax imagesare projected onto the left and right eyes of the observer in aparallax-barrier stereoscopic display device. FIG. 1 is across-sectional view as seen from above the head of the observer,showing a positional relationship in which both eyes (right eye 55R andleft eye 55L) of the observer are positioned on observational plane 30which is spaced from the display surface of the display device byoptimum observation distance OD and the center between the eyes of theobserver is aligned with the center of the display panel.

The display panel (not shown) comprises a group of light modulatingelements as a matrix of pixels (e.g., a liquid crystal panel). FIG. 1shows only pixels at both ends and the center of the display panel amongright-eye pixels 4R and left-eye pixels 4L which are alternatelyarrayed. Parallax barrier 6 which functions as an image swapping meansis disposed behind the display panel as viewed from the observer.Parallax barrier 6 is a barrier (light shield plate) with a number ofnarrow stripe-shaped vertical slits 6 a, and is arranged such that itslongitudinal direction is perpendicular to the direction along whichleft-eye pixels 4L and right-eye pixels 4R of the display panel arearrayed. A light source (not shown: so-called backlight) is disposedfurther behind the parallax barrier. Light emitted from the light sourcetravels through slits 6 a, has its intensity modulated by the pixels ofthe display panel, and is then projected toward the observer. Thedirections of light projected from right-eye pixels 4R and left-eyepixels 4L are limited by the presence of slits 6 a. The paths of lightrays which are emitted from slits 6 a and which travel through theclosest pixels are shown as light rays 20. These light rays 20 defineright-eye area 70R where images projected from all right-eye pixels 4Rare superposed and left-eye area 70L where images projected from allleft-eye pixels 4L are superposed. In right-eye area 70R, the observercan observe only the images projected from all right-eye pixels 4R. Inleft-eye area 70L, the observer can observe only the images projectedfrom all left-eye pixels 4L. Therefore, when right eye 55R of theobserver is positioned in right-eye area 70R and left eye 55L of theobserver is positioned in left-eye area 70L, the observer visuallyrecognizes parallax images projected onto the right and left eyesthereof as a stereoscopic image. Stated otherwise, when right eye 55R ofthe observer is positioned in right-eye area 70R and when left eye 55Lof the observer is positioned in left-eye area 70L, the observer canobserve a desired stereoscopic image.

The display device shown in FIG. 1 is designed such that all images(width P′) projected from right-eye pixels 4R and left-eye pixels 4L(width P) at the distance OD are superposed in order to maximize width Lof right-eye area 70R and left-eye area 70L. Width P′ of the projectedimages is determined mainly from distance h between slits 6 a and thepixels, pixel pitch P, and optimum observation distance OD. If P′ isincreased, then the width of right-eye area 70R and left-eye area 70L isalso increased, but the stereoscopic viewing area in which the observercan visually recognize stereoscopic images may not necessarily beincreased because it is impossible to place the eyes of the observer inany desired positions. If it is assumed that the distance between theeyes is represented by e, then the display device should preferably bedesigned such that P′ is equal to inter-eye distance e. If P′ is smallerthan inter-eye distance e, then the stereoscopic viewing area is limitedto P′. If P′ is greater than inter-eye distance e, then it is only thatan area in which both eyes are positioned in right-eye area 70R orleft-eye area 70L is increased. Minimum distance ND and maximum distanceFD up to the display panel, at which the observer can sec stereoscopicimages, are also determined by inter-eye distance c, right-eye area 70R,and left-eye area 70L.

As described above, the area in which the observer sees stereoscopicimages based on projected parallax images is determined by not onlyright-eye area 70R and left-eye area 70L which are optically determinedby the image swapping means, but also the inter-eye distance e of theobserver. Consequently, the stereoscopic viewing area may be expressedby an area around midpoint M between right eye 55R and left eye 55L ofthe observer.

As shown in FIG. 2, stereoscopic viewing area 71 thus defined is of adiamond-shaped rectangle. However, stereoscopic viewing area 71 shown inFIG. 2 is effective only when the plane including the eyes of theobserver and the surface of the display panel lie parallel to eachother.

FIG. 3 shows an optical model wherein parallax barrier 6 functioning asthe image swapping means is positioned in front of the display panel asviewed from the observer. As is the case with the example in whichparallax barrier 6 is positioned behind the display panel, the displaydevice is designed such that the observer is in optimum observationposition OD and the images (width P′) projected from the left and rightpixels (width P) are superposed. The paths of light rays which areemitted from the pixels and which travel through the closest slits 6 aare shown as light rays 20. These light rays 20 define right-eye area70R where images projected from all right-eye pixels 4R are superposedand left-eye area 70L where images projected from all left-eye pixels 4Lare superposed.

FIG. 4 shows a stereoscopic viewing area created by using a lenticularlens as an image swapping means.

FIG. 4 is similar to FIG. 3 except that the image swapping means isdifferent.

An optical model using a lenticular lens with the observer shifted outof the stereoscopic viewing area will be described below.

FIG. 5 is a cross-sectional view as seen from above the head of theobserver, showing the observer shifted to the right out of stereoscopicviewing area 71 which is expressed using midpoint M between right eye55R and left eye 55L. Right eye 55R of the observer is positionedoutside of right-eye area 70R, and left eye 55L is positioned withinright-eye area 70R. At this time, light rays 20 which are emitted fromleft-eye pixels 4L and right-eye pixels 4R and which travel through theprincipal points (vertexes) of closest cylindrical lenses 3 a do notreach the position of right eye 55R of the observer. Light rays 21 whichare emitted from left-eye pixels 4L and which travel through theprincipal points (vertexes) of second closest cylindrical lenses 3 a,define second left-eye area 72. In FIG. 5, the observer observes theimage projected from left-eye pixels 4L with right eye 55R, and observesthe image projected from right-eye pixels 4R with left eye 55L.Therefore, when the observer observes parallax images, the protrusiondepth is reversed (so-called pseudo-stereoscopic image), and theobserver fails to observe a desired stereoscopic image.

To solve the above problem, there has been proposed a process ofdetecting the position of the observer at all times and switching aroundthe displayed images of right-eye pixels and left-eye pixels dependingon the detected position (see, for example, Patent document 2).

There has also been proposed a process of capturing an image of anobserver with a camera, detecting a viewpoint position from an obtainedimage of the face of the observer, and adjusting parallax images (see,for example, Patent document 3).

For detecting a viewpoint position, there has been proposed a process ofdetecting a pupil with an infrared irradiator and a camera (see, forexample, Patent document 4).

PRIOR ART DOCUMENTS Patent Documents

-   Patent document 1: JP 2001-66547A (pages 3-4, FIG. 6)-   Patent document 2: JP 9-152668A-   Patent document 3: JP 2000-152285A-   Patent document 4: JP 11-72697A

Non-Patent Documents

-   Non-patent document 1: Nikkei Electronics, Jan. 6, 2003, No. 838, p.    26-27

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

While a portable stereoscopic display device allows the observer toadjust the display device to a position optimum for stereoscopic viewingusing the observer's own body, the display device itself may be tiltedor moved due to external factors such as operations on the displaydevice and swinging movements of the vehicle on which the display deviceis used.

As a result of movement of the display device, the positions of the eyesof the observer may be shifted out of the stereoscopic viewing area. Insuch a case, the observer may not only feel uncomfortable from viewingdual images and pseudo-stereoscopic images, but also feel tired fromrepeatedly viewing normal stereoscopic images, dual images, andpseudo-stereoscopic images, tending to suffer symptoms such as vertigoand motion sickness.

According to a general viewpoint tracking system, the display deviceneeds to incorporate a camera, an image processing function to detectviewpoint positions, and an infrared irradiator, and hence becomes largein size and has to meet requirements for sophisticated image processingcapabilities. Therefore, the general viewpoint tracking system is notsuitable for use on portable stereoscopic display devices.

It is an object of the present invention to provide a display device, aterminal device, and a display method which solve the above problems.

Means for Solving the Problems

According to the present invention, there is provided a display devicefor displaying an image, wherein movement of the display device isdetected and said image is displayed in a stereoscopic display or planardisplay depending on the detected movement.

According to the present invention, there is also provided a displaymethod for displaying an image on a display device, comprising:

detecting movement of the display device; and

displaying the image in either a stereoscopic display or a planardisplay depending on the detected movement.

Advantages of the Invention

According to the present invention, as described above, movement of thedisplay device is detected and an image is displayed in a stereoscopicdisplay or planar display depending on the detected movement. Therefore,even if the display device is moved against the will of the observer,placing the observer out of a stereoscopic viewing area, the observer iseasily prevented from observing a pseudo-stereoscopic image and a dualimage and hence is prevented from feeling uncomfortable and tired.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing an optical model of a parallax-barrierstereoscopic display device with a parallax barrier disposed behind adisplay panel;

FIG. 2 is a diagram showing a stereoscopic viewing area as an areaaround midpoint M between the right eye and the left eye of an observer;

FIG. 3 is a diagram showing an optical model of a parallax-barrierstereoscopic display device with a parallax barrier disposed in front ofa display panel;

FIG. 4 is a diagram showing an optical model of a stereoscopic displaydevice with a lenticular lens;

FIG. 5 is a diagram showing an optical model in which the observer isshifted out of a stereoscopic viewing area;

FIG. 6 is a front elevational view of a display device according to thepresent invention;

FIG. 7 is a cross-sectional view of the display device according to thepresent invention;

FIG. 8 is a functional block diagram of a display controller accordingto a first exemplary embodiment of the present invention;

FIG. 9(a) is a diagram showing image data according to first and secondexemplary embodiments of the present invention;

FIG. 9(b) is a diagram showing image data according to the first andsecond exemplary embodiments of the present invention;

FIG. 10 is a diagram showing an optical model in which the observerobserves in an optimum position parallax images on the display deviceaccording to the present invention;

FIG. 11 is a diagram showing an optical model in which the width ofimages projected from pixels is not equal to the inter-eye distance ofthe observer in the display device according to the present invention;

FIG. 12(a) is a diagram showing an optical model in which the distancebetween a central slit and an end slit of an image swapping means is notequal to the inter-eye distance of the observer in the display deviceaccording to the present invention;

FIG. 12(b) is a diagram showing an optical model in which the distancebetween a central slit and an end slit of an image swapping means is notequal to the inter-eye distance of the observer in the display deviceaccording to the present invention;

FIG. 13(a) is a diagram showing an optical model in which the displaydevice according to the present invention is moved along an X-axis;

FIG. 13(b) is a diagram showing an optical model in which the displaydevice according to the present invention is moved along the X-axis;

FIG. 14(a) is a diagram showing an optical model in which the displaydevice according to the present invention is moved along a Z-axis;

FIG. 14(b) is a diagram showing an optical model in which the displaydevice according to the present invention is moved along the Z-axis;

FIG. 15(a) is a diagram showing an optical model in which the displaydevice according to the present invention is moved along the X-axis andthe Z-axis;

FIG. 15(b) is a diagram showing an optical model in which the displaydevice according to the present invention is moved along the X-axis andthe Z-axis;

FIG. 16(a) is a diagram showing an optical model in which the displaydevice according to the present invention is tilted around a Y-axis;

FIG. 16(b) is a diagram showing an optical model in which the displaydevice according to the present invention is tilted around the Y-axis;

FIG. 17 is a view showing the display device according to the presentinvention with an acceleration sensor incorporated therein;

FIG. 18 is a diagram illustrative of the calculation of an angle of tiltbased on a gravitational acceleration at the time the display deviceaccording to the present invention in which the acceleration sensorincorporated therein is turned about the X-axis;

FIG. 19 is a diagram illustrative of the calculation of an angle of tiltbased on a gravitational acceleration at the time the display deviceaccording to the present invention in which the acceleration sensorincorporated therein is turned about the Y-axis;

FIG. 20 is a flowchart of an operation sequence of first, third, fourth,and sixth exemplary embodiments of the present invention;

FIG. 21 is a functional block diagram of the second exemplary embodimentof the present invention;

FIG. 22 is a flowchart of an operation sequence of second, third, fifth,and seventh exemplary embodiments of the present invention;

FIG. 23 is a functional block diagram of the third and fourth exemplaryembodiments of the present invention;

FIG. 24(a) is a diagram showing image data generated by an imagegenerator according to the third exemplary embodiment of the presentinvention;

FIG. 24(b) is a diagram showing image data generated by the imagegenerator according to the third exemplary embodiment of the presentinvention;

FIG. 25 is a functional block diagram of the third and fourth exemplaryembodiments of the present invention;

FIG. 26(a) is a diagram illustrative of a pixel structure of a displaypanel used in the fourth exemplary embodiment of the present invention;

FIG. 26(b) is a diagram illustrative of a pixel structure of a displaypanel used in the fourth exemplary embodiment of the present invention;

FIG. 27 is a diagram showing the positional relationship between thedisplay panel and a lenticular lens used in the fourth exemplaryembodiment of the present invention;

FIG. 28(a) is a diagram showing image data generated by an imagegenerator according to the fourth and fifth exemplary embodiments of thepresent invention;

FIG. 28(b) is a diagram showing image data generated by the imagegenerator according to the fourth and fifth exemplary embodiments of thepresent invention;

FIG. 28(c) is a diagram showing image data generated by the imagegenerator according to the fourth and fifth exemplary embodiments of thepresent invention;

FIG. 28(d) is a diagram showing image data generated by the imagegenerator according to the fourth and fifth exemplary embodiments of thepresent invention;

FIG. 29 is a diagram illustrative of a pixel structure of a displaypanel used in the sixth exemplary embodiment of the present invention;

FIG. 30 is a diagram showing an optical model in which images areprojected onto a plane which is spaced an optimum observation distanceaccording to the sixth exemplary embodiment of the present invention;

FIG. 31 is a diagram showing an optical model in which the right eye ofthe observer is positioned in area 74B and the left eye in area 74Caccording to the sixth exemplary embodiment of the present invention;

FIG. 32(a) is a diagram showing image data generated by an imagegenerator according to the sixth and seventh exemplary embodiments ofthe present invention;

FIG. 32(b) is a diagram showing image data generated by the imagegenerator according to the sixth and seventh exemplary embodiments ofthe present invention; and

FIG. 33 is a diagram showing an optical model in which the observerobserves parallax images in various combinations according to the sixthexemplary embodiment of the present invention.

MODES FOR CARRYING OUT THE INVENTION

Exemplary embodiments of the present invention will be described indetail below with reference to the drawings.

First Exemplary Embodiment

Description of the Structure

FIG. 6 is a front elevational view of a display device according to thepresent invention.

FIG. 7 is a cross-sectional view of the display device, taken along lineb of FIG. 6 as seen from above the head of the observer.

The display device according to the present invention includes displaypanel 11, image swapping means 13, display controller 12, and operationswitch 14 which are housed in casing 10.

Display panel 11 comprises a transmissive liquid crystal panel with amatrix of unit pixels. For displaying stereoscopic images, the unitpixels that are arrayed in a horizontal direction which is parallel tothe direction along which both eyes of the observer are arrayed arealternately used as left-eye pixels 4L and right-eye pixels 4R. In FIG.7, those left-eye pixels 4L and right-eye pixels 4R other than thepixels at the opposite ends and center of display panel 11 are omittedfrom illustration.

Image swapping means 13 is an electrooptical device for displaying aparallax barrier pattern, and may comprise a transmissive liquid crystalpanel, for example. Image swapping means 13 is placed over display panel11 so that when image swapping means 13 displays a parallax barrierpattern, transmissive regions acting as slits extend vertically withrespect to display panel 11 and are positioned between right-eye pixels4R and left-eye pixels 4L. The distance between image swapping means 13and display panel 11 and the pitch of the slits should preferably bedesigned such that when an optimum observation distance is determinedfor the observer, images projected from all right-eye pixels 4R ofdisplay panel 11 are projected onto right eye 55R of observer 50 andimages projected from all left-eye pixels 4L of display panel 11 areprojected onto left eye 55L of observer 50. When image swapping means 13does not display a parallax barrier pattern, it does not function as abarrier, and the images projected from both the right-eye pixels andleft-eye pixels are projected onto both eyes of the observer, as is thecase with the ordinary panel displays. Image swapping means 13 thuscontrols the projection of images displayed by display panel 11outwardly from display panel 11.

Display controller 12 has a function to drive display panel 11, afunction to control the barrier, and a function to detect movement ofcasing 10 for determining stereoscopic vision.

Display controller 12 will be described below with reference to FIG. 8.

FIG. 8 is a functional block diagram of display controller 12 accordingto a first exemplary embodiment of the present invention.

Display controller 12 comprises image generator 100, detector 80,judging section 90, display panel driving circuit 110, and imageswapping control circuit 111.

Detector 80 comprises a sensor for detecting a displacement caused whencasing 10 is moved. The displacement of casing 10 represents a change inan angle of tilt or a movement. If detector 80 comprises a sensor suchas an acceleration sensor, a geomagnetic sensor, or the like, thendetector 80 can calculate a displacement with respect to thegravitational acceleration or geomagnetism.

Judging section 90 includes memory 91 for storing information about anangle of tilt or a movement produced by the sensor of detector 80 andinformation about a stereoscopic viewing area of display panel II, andarithmetic unit 92 for determining whether or not both eyes of theobserver are in the stereoscopic viewing area from the informationobtained from the sensor of detector 80 and the information stored inmemory 91.

Image generator 100 has a function to generate image data to be sent todisplay panel 11, and comprises arithmetic unit 101, data storage 102,memory 103, and external IF (InterFace) 104. Image generator 100 alsohas a function to generate image data (3D data) with a parallax or imagedata (2D data) without a parallax depending on a signal from judgingsection 90.

Image data are generated by arithmetic unit 101 which reads data to bedisplayed from data storage 102 and performs an image processingsequence on the read data. Since the data to be displayed arethree-dimensional data including depth information, arithmetic unit 101should preferably perform a rendering process on the three-dimensionaldata to generate two-dimensional image data. 3D data used forstereoscopic display, i.e., two-dimensional image data for the left andright eyes having a parallax, are generated by setting two hypotheticalviewpoints corresponding to the left and right eyes of the observer andperforming a rendering process. 2D data for planar display, i.e., imagedata free of a parallax, are generated by setting one viewpointcorresponding to the center between the left and right eyes of theobserver and performing a rendering process. However, for stereoscopicdisplay of parallax images, the unit pixels of display panel 11 arealternately used as right-eye pixels and left-eye pixels. Therefore, thehorizontal resolution of two-dimensional image data to be generated forstereoscopic display is one half of display panel 11. Specifically, 3Ddata as image data to be generated are shown in FIG. 9(a), and 2D dataas image data to be generated are shown in FIG. 9(b).

As described above, image data should preferably be generated fromthree-dimensional data including depth information. However, data to bedisplayed which have been subjected to a rendering process may be storedin data storage 102 in advance and then may selectively be read fromdata storage 102. In other words, two-dimensional image corresponding toFIGS. 9(a) and 9(b) which do not include depth information may bestored, and selected depending on the stereoscopic display or planardisplay and read. According to this process, as no rendering process isrequired, arithmetic unit 101 may have a lower processing capability anda lower calculating rate than if a rendering process is used. Therefore,image generator 100 may be of an inexpensive configuration.

As described above, image generator 100 generates 2D/3D data dependingon a signal from judging section 90 and outputs the generated 2D/3D datato display panel driving circuit 110. At the same time, image generator100 has a function to send a signal which makes the barrier effectivefor stereoscopic display or which makes the barrier ineffective forplanar display, to image swapping control circuit 111.

Display panel driving circuit 110 has a function to generate signals(synchronizing signal, etc.) required to drive display panel 11. Imageswapping control circuit 111 has a function to generate a signal todisplay the parallax barrier pattern.

Image swapping means 13 may be an electrooptical device, which can beturned on and off by electric signals, comprising a lenticular lens madeup of a plurality of liquid crystal lenses, for example, rather than anelectrooptical device for turning on and off a parallax barrier pattern.

In FIG. 8, judging section 90 and image generator 100 include respectiveindependent arithmetic units 92, 101 for illustrative purposes. However,judging section 90 and image generator 100 may share one arithmeticunit. Alternatively, the processing functions may be provided in anarithmetic unit for performing other functions (e.g., a communicationcontrol function) of the portable display device to which the presentinvention is applied, or may be provided in another processor.

A process of determining whether or not both eyes of the observer arepositioned in the stereoscopic viewing area will be described below withreference to the drawings. In the description which follows, the imageswapping means comprises an electrooptical device for displaying aparallax barrier pattern. However, the image swapping means may comprisea lenticular lens as described above. If the image swapping meanscomprises a lenticular lens, then the parallax barrier lens may bereplaced with the lenticular lens and the slits may be replaced with theprincipal points of the lenses of the lenticular lens in the descriptionwhich follows.

FIG. 10 shows an optical model in which observer 50 observes parallaximages on the display device according to the present invention.

For illustrative purposes, an XYZ orthogonal coordinate system isdefined as follows: A horizontal direction of display panel 11 alongwhich both eyes of observer 50 are arrayed is defined as an X-axis. Adirection which is perpendicular to the projected plane (a plane onwhich a matrix of parallel pixels is present) of the display device andthe X-axis is defined as a Y-axis. An axis which perpendicularly crossesthe projected plane of the display device is defined as a Z-axis.Positive and negative directions along the X-, Y-, Z-axes are defined asshown in FIG. 10.

In FIG. 10, it is assumed that observer 50 and the display device are ina positional relationship optimum for stereoscopic vision, the distancefrom image swapping means 13 to both eyes 55R, 55L of the observerserves as optimum observation distance OD, and the XY plane which isspaced distance OD from image swapping means 13 serves as an optimumobservation plane.

Image swapping means 13 is functioning as a parallax barrier, and have acentral slit and end slits which are spaced a part from each other bydistances WS.

Display panel 13 includes a plurality of unit pixels and uses themalternately as left-eye pixels 4L and right-eye pixels 4R in the X-axisdirection. However, only left-eye pixel 4L and right-eye pixel 4R at thecenter are illustrated. If the pitch (width) of unit pixels isrepresented by P, then the width at the optimum projection plane of animage projected from a slit which is positioned at a shortest distancefrom each pixel is represented by P′. Light rays which form images P′projected from left-eye pixels and right-eye pixels at opposite ends andcenter of display panel 11 are represented by 22R, 23R, 24R, 25R, 22L,23L, 24L, 25L.

As shown in FIG. 10, display panel 11 and image swapping means 13 shouldpreferably be designed such that images P′ projected from all theright-eye pixels are superposed at the optimum projection plane andimages P′ projected from all the left-eye pixels are superposed at theoptimum projection plane. P′ should preferably be set to be equal tointer-eye distance e of the observer.

Optimum projection plane OD is of a designed value. Under the abovedesign conditions, right-eye area 70R and left-eye area 70L, for theobserver to have appropriate stereoscopic vision, are determined fromlight rays 22R, 23R, 24R, 22L, 23L, 24L emitted from the slits at theopposite ends of the image swapping means, as shown in FIG. 10.

FIG. 11 shows an optical model in which width P′ of the images projectedfrom the pixels is not equal to inter-eye distance e of the observer.

As described above in the background art, the stereoscopic viewing areais smaller when P′<e. When P′>e, right-eye area 70R and left-eye area70L can be greater. However, since it is impossible to place the eyes ofthe observer in any desired positions, the stereoscopic viewing area isnot widened due to the limitation of the inter-eye distance. If P′<e,then the distance between the pixels and the parallax barrier may belarge, resulting in an advantage in which there is an increased choiceof components in the designing of the display device. If P′>e, then whenthe observer is shifted from an optimum observational position, it ispossible to reduce an area in which opposite parallel images areprojected onto the left and right eyes, resulting in a reversedprotrusion depth (so-called pseudo-stereoscopic image). In thedescription which follows, display panel 11 with P′=e is used for anoptical model. However, a display panel with P′<e or P′>e may also beused in the present invention.

FIG. 10 shows an optical model in which WS and width P′ of projectedimages are equal to each other, i.e., WS and inter-eye distance e of theobserver are equal to each other.

FIG. 12 shows optical models in which WS>e (=P′) and WS<e (=P′). FIG.12(a) shows an optical model in which WS>e, and FIG. 12(b) shows anoptical model in which WS<e.

Right-eye area 70R and left-eye area 70L for observer 50 to be able tohave stereoscopic vision are narrower in the front-back direction forWS>e and wider in the front-back direction for WS<e when they aredesigned having the same optimum observation distance OD. Based on theaverage inter-eye distance of adult people, inter-eye distance e shouldsuitably be designed in the range from 62 mm to 65 mm. If e=63 mm, thenthe stereoscopic display device shown in FIG. 10 has a horizontaldimension of WS.times.2=126 mm. In view of the size of portable displaydevices, either one of relationships WS>e, WS=e, WS<e is applicable. Inthe description which follows, the optical model with WS=e is used forillustrative purposes.

As shown in FIGS. 10 through 12(a), 12(b), right-eye area 70R in whichthe projected images for the right-eye are superposed and left-eye area70L in which the projected images for the left-eye are superposed aredetermined based on the design conditions. Though the sizes of the areasvary depending on the design conditions, the areas of the completeddisplay device do not vary when it displays stereoscopic images, but areinherent in the display device. If right-eye area 70R and left-eye area70L which are derived from the design conditions, or right-eye area 70Rand left-eye area 70L which are measured from the completed displaydevice are stored as data of the stereoscopic viewing area, then data ofthe positions of both eyes of the observer are acquired, and the storeddata and the acquired data are compared with each other by thearithmetic unit, which then determines whether or not both eyes of theobserver are positioned in the stereoscopic viewing area. Thestereoscopic viewing area and the positions of both eyes of the observerare relatively related to each other. Therefore, if both eyes of theobserver are not moved from the optimum observational position, then itis determined whether or not stereoscopic vision is possible dependingon movement of display device casing 10. The display device according tothe present invention determines whether or not both eyes of theobserver are positioned in the stereoscopic viewing area by storing thedata of the stereoscopic viewing area and detecting movement of displaydevice casing 10.

For judging stereoscopic vision, it is preferable to store the boundaryinformation of the diamond shapes of right-eye area 70R and left-eyearea 70L. The boundary information of the diamond shapes shown in FIGS.10 through 12(a), 12(b) is determined by light rays 22R, 23R, 24R, 22L,23L, 24L shown in FIG. 10.

However, if all images P′ projected from the left-eye pixels and theright-eye pixels are not superposed, for example, then light rays 25R,25L shown in FIG. 10 represent the boundaries of areas 70R, 70L,respectively. Depending on the size of an area (3D crosstalk area) inwhich the projected images for the left and right eyes are superposed tocause the observer to view a dual image, the boundary information canalso be determined by light rays in view of the size of such an area. Asa consequence, the boundary information may be representative of apolygonal shape other than the diamond shape. However, it is possible tojudge stereoscopic vision without the boundary information of apolygonal shape which represents the information of the left-eye areaand the right-eye area, depending on the design conditions of thedisplay device. An example of such a judging process will be describedbelow.

The positional relationship between casing 10 and observer 50 shown inFIG. 10 is regarded as representing the optimum observational position,and the range in which stereoscopic vision is possible when observer 50is not moved, but when casing 10 is moved will be described below withreference to the drawings.

FIG. 13 is a set of diagrams showing limitations on stereoscopic visionat the time casing 10 is moved parallel to the X-axis.

FIG. 13(a) is a diagram showing an optical model in which casing 10 ismoved in a positive (+) direction along the X-axis, and FIG. 13(b) is adiagram showing an optical model in which casing 10 is moved in anegative (−) direction along the X-axis. Observer 50 is able to haveappropriate stereoscopic vision when right eye 55R is in right-eye area70R and left eye 55L is in left-eye area 70L. Therefore, the distance ofmovement in the positive (+) direction along the X-axis is limited whenlight rays 22R, 23R emitted from the display device are aligned withboth eyes of the observer. The distance of movement in the negative (−)direction along the X-axis is limited when light rays 22L, 23L emittedfrom the display device are aligned with both eyes of the observer.

FIG. 14 is a set of diagrams showing limitations on stereoscopic visionat the time casing 10 is moved parallel to the Z-axis.

FIG. 14(a) is a diagram showing an optical model in which casing 10 ismoved in a positive (+) direction along the Z-axis, and FIG. 14(b) is adiagram showing an optical model in which casing 10 is moved in anegative (−) direction along the Z-axis. The distance of movement in thepositive (+) direction along the Z-axis is limited when light rays 23R,23L emitted from the display device are aligned with both eyes of theobserver. The distance of movement in the negative (−) direction alongthe Z-axis is limited when light rays 22L, 22L emitted from the displaydevice are aligned with both eyes of the observer.

FIG. 15 is a set of diagrams showing limitations on stereoscopic visionat the time casing 10 is moved in the directions of the X- and Z-axeswhile remaining parallel to the surface of display panel 11 and theobservation plane including both eyes of observer 50.

FIG. 15(a) is a diagram showing an optical model in which casing 10 ismoved in the positive (+) direction along the X-axis and the positive(+) direction along the Z-axis. The limitation on appropriatestereoscopic vision by observer 50 is reached when light ray 23R isaligned with right eye 55R. FIG. 15(b) is a diagram showing an opticalmodel in which casing 10 is moved in the negative (−) direction alongthe X-axis and the negative (−) direction along the Z-axis. Thelimitation on appropriate stereoscopic vision by observer 50 is reachedwhen light ray 22R is aligned with right eye 55R.

The conditions for limiting stereoscopic vision have been describedabove with reference to FIGS. 13(a), 13(b) through 15(a), 15(b) in whichWS=e. If WS<e and WS>e, light rays 24R, 24L do not contribute to theconditions for limiting stereoscopic vision unless the angle formedbetween the Z-axis and light ray 24R or 24L is greater than the angleformed between the Z-axis and light ray 22R or 22L in the XZ plane. Theconditions limiting stereoscopic vision are given in Table 1 below.

TABLE 1 Movement of Movement along Z-asis: Movement along Z-axis: casing10 positive (+) negative (−) Movement Light ray 23R is aligned Light ray22L is aligned along X-axis; with right eye 55R wtih left eye 55Lpositive (+) Movement Light ray 23L is aligned Light ray 22R is alignedalong X-axis; with left eye 55L with right eye 55R negative (−)The tilt of light rays 22R, 23R, 24R, light rays 22L, 23L, 24L withrespect to the display surface is determined when the stereoscopicdisplay device is designed. Consequently, it is possible to makecalculations to determine whether or not stereoscopic vision is possibleonce the distance that casing 10 moves from the optimum observationalposition is known.

The above conditions apply when casing 10 is not tilted, i.e., when thesurface of display panel 11 and the plane on which both eyes of theobserver are positioned remain parallel to each other. When casing 10 istilted, the limitation on stereoscopic vision needs to be calculatedtaking into account the angle of tilt of casing 10.

FIG. 16 is a set of diagrams showing limitations on stereoscopic visionat the time casing 10 is tilted about the Y-axis on the surface ofdisplay panel 11. FIG. 16(a) shows an optical model in which casing 10is turned to the left about the Y-axis with its positive (+) directiontoward the viewer of FIG. 16(a). The limitation on appropriatestereoscopic vision by observer 50 is reached when light ray 23L isaligned with left eye 55L. FIG. 16(b) shows an optical model in whichcasing 10 is turned to the right about the Y-axis with its positive (+)direction toward the viewer of FIG. 16(a). The limitation on appropriatestereoscopic vision by observer 50 is reached when light ray 23R isaligned with right eye 55R. Since the tilt of light rays 23R, 23L isdetermined when the display device is designed, it is possible to makecalculations to determine whether or not stereoscopic vision is possibleonce the angle of tilt of casing 10 from the optimum observationalposition is known.

As described above, it is possible to judge stereoscopic vision based onthe distance of movement and the angle of tilt of casing 10 from theoptimum observational position, and the angles of light rays 22R, 22L,23R, 23L with respect to the display panel surface which are determinedwhen the display device is designed.

Specific detecting means of angle-of-tilt detector 81 anddistance-of-movement detector 82 of detector 80 will be described below.

A three-axis acceleration sensor used as an example of the angle-of-tiltdetector and the distance-of-movement detector will be described below.

Output data from an acceleration sensor include various signalsindicative of qualities other than an angle of tilt and the distance ofmovement to be ascertained. Major ones of those signals arerepresentative of an acceleration component directed toward the earthaxis by the gravitational acceleration and a noise component caused byenvironmental factors such as vibrations which are simultaneouslyapplied to the human body, which is holding the casing, and to thecasing itself. The noise component caused by environmental factors suchas vibrations can effectively be removed by a filter, most preferably adigital filter. Depending on the characteristics of the environment andthe user, it is effective to use a filter which utilizes characteristicsin the frequency domain by way of Fourier transform or wavelettransform. A process of detecting a signal which has been processed bythe above filtering process will be described below.

FIG. 17 shows an acceleration sensor incorporated in casing 10.

A coordinate system of lenticular lens 3 of the display panel and acoordinate system of the acceleration sensor are defined as shown inFIG. 17. Specifically, the observer is positioned in the positivedirection (indicated by the arrow) along the Z-axis, and observes thedisplay panel that is present in the negative direction along theZ-axis. The positive direction along the Y-axis represents an upwarddirection of the display panel, and the negative direction along theY-axis represents a downward direction of the display panel. The displaypanel is often used obliquely to the vertical direction along the earthaxis, as shown in FIG. 18.

FIG. 18 is a diagram as seen from a plane including the Y-axis and theZ-axis shown in FIG. 17. The angle of tilt between the display panel andthe vertical direction along the earth axis is represented by φ, and thevector of the gravitational acceleration by G.

The distance of movement can be calculated by calculating the speed byintegrating the output from the acceleration sensor with respect to timeand then by integrating the calculated speed with respect to time.However, it is necessary to pay attention to two points. The first pointis concerned with the accumulation of noises caused by the integratingprocess, and the second point is about the effect that the gravitationalacceleration has.

First, the first point about an accumulation of noises will be describedbelow. If a noise is introduced into the detected acceleration, then thespeed or the distance of movement changes greatly due to the integratingprocess even though the noise may be a single shot. Specifically, when asignal with a single noise a introduced therein is integrated, the speedsubsequent to the noise changes only by αΔt where Δt indicates the timefrom the introduction of the noise to the end of the integratingprocess. In the calculation of the distance of movement, it varies byα(Δt)2 after the noise is introduced. Therefore, in particular, thedistance of movement varies greatly due to the integrating process.

Two processes to be described below are effective to handle the noise.The first process uses a filter for smoothing noise. The second processshortens the integrating time. Specifically, if the integrating time isreduced, Δt is reduced, resulting in a reduction in the variation of thedistance of movement due to noise. By adding the reduced distance ofmovement produced in the reduced integrating time, the distance ofmovement can be calculated in a desired time.

The second point about the gravitational acceleration will be describedbelow. Since the gravitational acceleration is present at all times, itis introduced into the outputs of all acceleration sensors. In order toeliminate the effect that the gravitational acceleration has, initialoutputs ax0, ay0, az0 of the acceleration sensor are recorded, and onlythe differences between subsequent outputs of the acceleration sensorand the initial outputs of the acceleration sensor are used in theintegrating process.

The above process makes it possible to calculate the distance ofmovement without being affected by the gravitational acceleration. Ifthere is no rotation about the Z-axis, then the gravitationalacceleration does not affect ax. Therefore, if no rotation about theZ-axis is observed, only initial outputs ay0, az0 may be recorded andthe differences may be taken to calculate the distance of movement moresimply.

As to the angle of tilt, the display panel may be tilted in variousways. The display panel may pitch, roll, and yaw about respectivecoordinate axes as is the case with airplanes and cars. Since anytilting movement can be expressed by a combination of pitching, rolling,and yawing, all tilting movements can easily be analyzed by analyzingbasic pitching, rolling, and yawing movements.

The relationship between a coordinate system and pitching, rolling, andyawing is defined as follows: Pitching is defined as rotation about theX-axis. Specifically, pitching refers to rotation of the display panelin a direction to bring the upper end (+Y) thereof toward the observeror in a direction to bring the lower end (−Y) thereof toward theobserver. Rolling is defined as rotation about the Y-axis. Specifically,rolling refers to rotation of the display panel in a direction to bringthe right end (+X) thereof toward the observer or in a direction tobring the left end (−X) thereof toward the observer. Yawing is definedas rotation about the Z-axis. Specifically, yawing refers to rotation ofthe display panel about the direction of view of the observer within aplane which faces the observer.

Pitching can be determined as follows:

FIG. 18 shows a plane including the Y-axis and the Z-axis andgravitational acceleration G in that plane.

The display panel is displaced only about the X-axis. An accelerationsensor in the Y-axis direction detects a component of the gravitationalacceleration along the Y-axis, i.e., detects −G cos(φ) which is acomponent of the gravitational acceleration that is mapped onto theY-axis. When the observer holds casing 10 in an attitude that is easyfor stereoscopic vision while at rest, the output of the accelerationsensor is stored as an initial value representative of the component ofthe gravitational acceleration that is mapped onto the Y-axis. If theoutput of the acceleration sensor along the Y-axis direction in theinitial state that is easy for observation is represented by ay0, thenay0=−G cos(φ). Since the gravitational acceleration is of asubstantially constant value on the ground, a pitch angle φ0 in theinitial state that is easy for observation is determined asφ0=arccos(−ay0/G).

Similarly, a pitch angle φ at the time that the angle of tilt is changedis given as φ=arccos(−ay/G) using the output ay of the accelerationsensor at the time. The pitch angle φ makes it possible to obtain achange from the pitch angle φ0 in the initial state and a change in, thepitch angle from time to time.

Rolling, which greatly affects the visibility of stereoscopic vision,can be determined in the same manner as with pitching. In this case, thedisplay panel is displaced only about the Y-axis. A gravitationalacceleration component in the Y-axis direction is the same as that shownin FIG. 18 and is represented as −G cos(β). A gravitational accelerationcomponent in the Z-axis direction is represented as −G sin(φ) in FIG. 18where there is no rotation about the Y-axis. If there is rotation aboutthe Y-axis, however, then the gravitational acceleration is divided intoa component in the Z-axis direction and a component in the X-axisdirection, as shown in FIG. 19.

In FIG. 19, an X-axis and a Z-axis which are indicated by the brokenlines represent the axis directions before the display panel rotatesabout the Y-axis. At this time, a component of the gravitationalacceleration which is perpendicular to the Y-axis extends in thenegative direction along the Z-axis and is represented as −G sin(φ).When the display panel rotates about the Y-axis through an angle β, theX-axis and the Z-axis are displaced respectively to an X′-axis and aZ′-axis shown in FIG. 19. A component of the gravitational accelerationon the Z′-axis is represented by −Gin(φ)cos(β). If an initial state inthe Z-axis direction is determined in advance in the same manner as whenthe initial state is determined for pitching, then −G sin(φ) whichrepresents a gravitational acceleration component on the Z-axis for β=0is detected as the output az0 of an acceleration sensor in the Z-axisdirection. A roll angle β at the time the angle of tilt is changed isgiven as β=arccos {−az/[G sin(φ)]} using the output az of theacceleration sensor at the time. The roll angle β makes it possible toobtain a change in the roll angle from time to time.

While the three-axis acceleration sensor has been described above by wayof example in the present exemplary embodiment, pitching and rolling canobviously be detected by a two-axis acceleration sensor.

The process of detecting an angle of tilt and the distance of movementhas been described above by way of example. An angle of tilt may bedetected by a geomagnetic sensor, and a distance of movement may bedetected by an acceleration sensor. A process of detecting an angle oftilt with a three-axis geomagnetic, sensor is similar to the aboveprocess of detecting an angle of tilt with the acceleration sensorexcept that the gravitational acceleration is replaced withgeomagnetism. An angle of tilt may further be detected by an angularvelocity sensor or a gyrosensor, and a distance of movement may furtherbe detected by a small-size camera or an ultrasonic transmission sourceand an ultrasonic sensor.

[Description of the Operation]

Operation of the present exemplary embodiment will be described belowwith reference to a flowchart shown in FIG. 20.

At the same time that stereoscopic display is started, the sensor fordetecting movement of casing 10 is activated.

Then, a reference screen for guiding the observer to the optimumobservational position is displayed. The stereoscopic display accordingto the present exemplary embodiment refers to a process of turning onthe function of the image swapping means (e.g., to display a parallaxbarrier pattern), sending image data with a parallax as shown in FIG.9(a) to display panel 11, and projecting images respectively onto theleft and right eyes of the observer.

In step 1, the observer adjusts the position and tilt of casing 10 sothat the displayed reference screen can be seen as a stereoscopic image.

Then, in step 2, with the position and tilt of casing 10 being adjustedby the observer, an output from detector 80 is recorded as an initialvalue, and desired contents are played back for stereoscopic display.

In step 3, a distance of movement and an angle of tilt in prescribedperiod ΔT are calculated from an output from detector 80 and the initialvalue.

In step 4, stereoscopic vision is judged based on the distance ofmovement and the angle of tilt which have been calculated. Specifically,stereoscopic vision is judged based on whether the distance of movementand the angle of tilt which have been calculated are greater thanrespective preset threshold values. For example, if the calculateddistance of movement is smaller than the preset distance-of-movementthreshold value, then it is judged that stereoscopic vision is possible.If the calculated angle of tilt is smaller than the preset angle-of-tiltthreshold value, then it is judged that stereoscopic vision is possible.If it is judged that stereoscopic vision is possible, then stereographicdisplay is performed in step 5, from which control goes to step 7.

If it is judged that stereoscopic vision is not possible, thenstereographic display switches to planar display in step 6. The planardisplay according to the first exemplary embodiment refers to a processof turning off the function of the image swapping means (e.g., to notdisplay a parallax barrier pattern), sending image data with no parallaxas shown in FIG. 9(b) to display panel 11, and projecting aparallax-free image onto the observer. After stereographic displayswitches to planar display, control goes back to step 3 in which adistance of movement and an angle of tilt in prescribed period ΔT arecalculated.

In step 7, it is determined whether the initial value for use as areference in calculating movement of casing 10 is to be updated or not.If “No” is judged in step 7, then control goes back to step 3. If “Yes”is judged in step 7, control goes back to step 2 in which an output fromdetector 80 at this time is recorded in place of the initial valuerecorded in step 1.

The above steps are repeated.

In the above operation flow, prescribed period ΔT should preferably beset to a value between about the frame cycle of display panel 11 andabout 0.2 second. As can be seen from the flowchart shown in FIG. 20, ifΔT is long, then switching from the stereoscopic display to the planardisplay is delayed with respect to movement of casing 10. As a result,the stereoscopic display switches to the planar display after theobserver sees a pseudo-stereoscopic image and a dual image.

Therefore, it is better to have shorter ΔT. However, even if switchingbetween the stereoscopic display and the planar display is to be made aplurality of times within one frame cycle of display panel 11, there isnot enough time to switch image data for the entire display screen. Inother words, ΔT that is shorter than the frame cycle is not effectiveenough for switching following with respect to movement of casing 10.

Step 7 serves as a function to deal with a change in the position andtilt of the casing which happens when the observer changes its attitudeor changes the way in which the observer holds the display device.Therefore, the judging process in step S7 does not need to be carriedout for each pass. The number of passes may be counted, and when anappropriate count is reached, the observer may be prompted to enter ajudgement using an operation switch or the like on the display device,or when a prescribed count is reached, an automatic decision “Yes” maybe made. However, if an acceleration sensor is used to detect a distanceof movement, then it is preferable to update the initial value becauseit acts to clear an accumulated error.

As described above with reference to FIGS. 13(a), 13(b) through 16(a),16(b), the conditions for judging stereoscopic vision in step 4 are theconditions for limiting stereoscopic vision which are derived fromright-eye area 70R and left-eye area 70L that are determined at the timethe display device is designed and from the optimum positions of botheyes of the observer that are also determined at the time the displaydevice is designed. The conditions for judging stereoscopic vision whichare derived from the above designing conditions may have a function,which is applied to initial settings, to allow the observer to move andtilt casing 10 to look for limitations on stereoscopic vision and storeconditions for limiting stereoscopic vision, while performingstereoscopic vision in step 1 (a function to record a distance ofmovement and an amount of tilt or the output of a relevant sensor at thetime stereoscopic vision is limited). In this case, though the observeris burdened, the inter-eye distance of the observer and theobservational distance which is preferred by the observer, rather thanthe inter-eye distance and the observational distance which arerepresentative of design parameters of the display device, arereflected, making it possible to switch between stereoscopic display andplanar display in a manner to match the individual observer.Furthermore, if the conditions for judging stereoscopic vision furtherhave a function to save the recorded conditions for limitingstereoscopic vision when the display device is switched off, then it isnot necessary to perform a process of recording the conditions forlimiting stereoscopic vision each time the observer uses the displaydevice.

Second Exemplary Embodiment

A second exemplary embodiment is of the same structure as the firstexemplary embodiment described above, and uses the same method ofdetermining whether or not both eyes of the observer are in thestereoscopic viewing area as the first exemplary embodiment. However,the second exemplary embodiment is different from the first exemplaryembodiment as to its operation after both eyes of the observer arejudged as being positioned outside of the stereoscopic viewing area andstereoscopic display switches to a planar display until stereoscopicdisplay is performed again. Specifically, after stereoscopic displayswitches to planar display, stereoscopic display will be resumed whenthe position and tilt of casing 10 returns values near the recordedinitial values. The values near the initial values for resumingstereoscopic display (hereinafter referred to as 2D.fwdarw.3D returnvalues) should preferably be selected by the observer based on itspreference from a choice of large/medium/small values disposed on thedisplay screen (for example, “large value” may be .+−.10% of the initialvalues, “medium values” may be .+−.5% of the initial values, and “smallvalues” may be .+−.2% of the initial values). Therefore, the secondexemplary embodiment is different as to the operation from the firstexemplary embodiment because of the added function to set the2D.fwdarw.3D return values.

FIG. 21 is a functional block diagram of the second exemplary embodimentof the present invention. As with the first exemplary embodiment, thesecond exemplary embodiment comprises display panel 11, image swappingmeans 13, and display controller 12. Display controller 12 comprisesimage generator 100, detector 80, judging section 90, display paneldriving circuit 110, and image swapping control circuit 111. As shown inFIG. 21, the second exemplary embodiment is the same as the firstexemplary embodiment except that 2D.fwdarw.3D return value settingsection 93 is added to judging section 90. The process of determiningwhether or not both eyes of the observer are in the stereoscopic viewingarea is also the same as the first exemplary embodiment.

With respect to the 2D.fwdarw.3D return values, judging areas for thereturn values which are formed by reducing right-eye area 70R andleft-eye area 70L shown in FIG. 10 around the right-eye and left-eyepositions of an optimum observer may be calculated and used forjudgment.

Operation of the second exemplary embodiment will be described belowwith reference to a flowchart shown in FIG. 22.

At the same time that the stereoscopic display is started, the sensorfor detecting movement of casing 10 is activated.

Then, a reference screen for guiding the observer to the optimumobservational position is displayed. The stereoscopic display accordingto the second exemplary embodiment refers to a process of turning on thefunction of the image swapping means (e.g., to display a parallaxbarrier pattern), sending image data with a parallax as shown in FIG.9(a) to the display panel, and projecting images respectively onto theleft and right eyes of the observer, as with the first exemplaryembodiment.

In step 11, the observer adjusts the position and tilt of casing 10 sothat the displayed reference screen can be seen as a stereoscopic image.The observer also generates 2D.fwdarw.3D return values for switchingfrom planar display to stereoscopic display with 2D.fwdarw.3D returnvalue setting section 93.

Then, in step 12, with the position and tilt of casing 10 being adjustedby the observer, an output from detector 80 is recorded as an initialvalue, and the desired content are played back for stereoscopic display.

In step 13, a distance of movement and an angle of tilt in prescribedperiod ΔT are calculated from an output from detector 80 and the initialvalue.

In step 14, stereoscopic vision is judged based on the distance ofmovement and the angle of tilt which have been calculated. Specifically,stereoscopic vision is judged based on whether the distance of movementand the angle of tilt which have been calculated are greater thanrespective preset threshold values. For example, if the calculateddistance of movement is smaller than the preset distance-of-movementthreshold value, then it is judged that stereoscopic vision is possible.If the calculated angle of tilt is smaller than the preset angle-of-tiltthreshold value, then it is judged that stereoscopic vision is possible.If it is judged that stereoscopic vision is possible, then stereographicdisplay is performed in step 15, from which control goes to step 17.

If it is judged that stereoscopic vision is not possible, thenstereographic display switches to planar display in step 16. The planardisplay according to the second exemplary embodiment refers to a processof turning off the function of the image swapping means (e.g., to notdisplay a parallax barrier pattern), sending image data with no parallaxas shown in FIG. 9(b) to the display panel, and projecting aparallax-free image onto the observer, as with the first exemplaryembodiment.

After stereographic display switches to planar display, control goes tostep 18 in which a distance of movement and an angle of tilt inprescribed period ΔT are calculated. Then, in step 19, it is determinedwhether the distance of movement and the angle of tilt which arecalculated fall within the 2D.fwdarw.3D return values that have beenset. If the distance of movement and the angle of tilt fall within the2D.fwdarw.3D return values, then planar display switches tostereographic display in step 15. If the distance of movement and theangle of tilt do not fall within the 2D.fwdarw.3D return values, thenplanar display remains unchanged and control goes back to step 18. Inother words, unless the distance of movement and the angle of tilt fallwithin the 2D.fwdarw.3D return values, step 18 and step 19 are repeatedand planar display does not switch back to stereographic display.

If the output from detector 80 falls back within the 2D.fwdarw.3D returnvalues, then planar display switches to stereographic display andthereafter control goes to step 17.

In step 17, it is determined whether the initial value for use as areference in calculating movement of casing 10 is to be updated or not.If “No” is judged in step 7, then control goes back to step 13. If “Yes”is judged in step 17, control goes back to step 12 in which an outputfrom detector 80 at this time is recorded in place of the initial valuerecorded in step 11.

The above steps are repeated.

In the above operation flow described with reference to FIG. 22,prescribed period ΔT should preferably be set to a value between, aboutthe frame cycle of display panel 11 and about 0.2 second, as describedabove in the operation of the first exemplary embodiment. Furthermore,the judging process in step S17 does not need to be carried out for eachpass, as described above in the operation of the first exemplaryembodiment. The number of passes may be counted, and when an appropriatecount is reached, the observer may be prompted to enter a judgementusing an operation switch or the like on the display device, or when aprescribed count is reached, an automatic decision “Yes” may be made.Moreover, the conditions for judging stereoscopic vision in step 14 mayhave a function to allow the observer to determine and store conditionsfor limiting stereoscopic vision, as described above in the operation ofthe first exemplary embodiment.

As described above, the second exemplary embodiment has complexprocessing and additional functions compared with the first exemplaryembodiment. However, since the observer sets points to return tostereoscopic display on its own, the second exemplary embodiment iseffective for reducing a feeling of strangeness at the time that planardisplay switches back to stereoscopic display and also for reducing anuncomfortable feeling caused when frequent switching occurs between thestereoscopic display and planar display.

Third Exemplary Embodiment

A third exemplary embodiment resides in using an ordinary optical device(a parallax barrier, a lenticular lens, or the like) as the imageswapping means, rather than the electrooptical device that can be turnedon and off by electric signals (e.g., a transmissive liquid crystalpanel for displaying a parallax barrier pattern) used in the first andsecond exemplary embodiments. The other configurations than the imageswapping means are the same as those of the first exemplary embodiment.

FIG. 23 is a functional block diagram of the third exemplary embodiment.

As with the first exemplary embodiment, the third exemplary embodimentcomprises display panel 11, image swapping means 13, and displaycontroller 12. Display controller 12 is similar to the displaycontroller (see FIG. 8) according to the first exemplary embodimentexcept that it dispenses with image swapping control circuit 111.

As shown in FIG. 23, display controller 12 according to the thirdexemplary embodiment comprises image generator 100, detector 80, judgingsection 90, and display panel driving circuit 110. The roles of thesecomponents are the same as those of the first exemplary embodiment, andwill not be described below.

However, 2D data used for the planar display which are generated byimage generator 100 are different from those in the first exemplaryembodiment. According to the third exemplary embodiment, the imageswapping function cannot be turned off. For the planar display as wellas the stereoscopic display, therefore, the unit pixels of the displaypanel are alternately used as right-eye pixels and left-eye pixels.Therefore, the horizontal resolution of two-dimensional image data to begenerated for planar display is also one half of the display panel.Image data should preferably be generated by performing a renderingprocess on three-dimensional data including depth information. 3D dataused for stereoscopic display are generated by setting two hypotheticalviewpoints corresponding to both eyes of the observer and performing arendering process. 2D data for the planar display are generated bysetting one viewpoint corresponding to the center between both eyes ofthe observer and performing a rendering process. Specifically, 3D dataas image data to be generated are shown in FIG. 24(a), and 2D data asimage data to be generated are shown in FIG. 24(b).

As with the first exemplary embodiment, image data should preferably begenerated from three-dimensional data including depth information.However, data to be displayed which have been subjected to a renderingprocess, as shown in FIGS. 24(a) and 24(b) may be stored in data storage102 in advance and then may be selectively read from data storage 102.According to this process, as no rendering process is required,arithmetic unit 101 may have a lower processing capability and a lowercalculating rate than if a rendering process is required. Therefore,image generator 100 may be of an inexpensive configuration.

Since the third exemplary embodiment has many features in common withthe first exemplary embodiment, only the differences will be describedbelow with reference to the flowchart shown in FIG. 20 which shows theoperation sequence according to the first exemplary embodiment.

According to the third exemplary embodiment, the image swapping means isnot controlled at the time of switching to planar display in step 6. Forthe planar display according to the third exemplary embodiment, aright-eye image shown in FIG. 24(a) is sent to the left- and right-eyepixels, or a left-eye image is sent to the left- and right-eye pixels,or image date shown in FIG. 24(b) are sent to project an image free of aparallax onto the observer. In other words, the image data sent to theleft- and right-eye pixels are identical to each other.

For planar display in step 16, using the coordinate system describedwith reference to FIGS. 10, 13(a), 13(b) through 16(a), 16(b), left-eyedata may be sent to the left- and right-eye pixels when casing 10 isjudged as being moved in the negative direction along the X-axis ortilted to the left, and left-eye data may be sent to the left- andright-eye pixels when casing 10 is judged as being moved in the positivedirection along the X-axis or tilted to the right.

By switching between the data as described above, it is possible toreduce a feeling of strangeness at the time that a stereoscopic displayswitches to a planar display. According to our research, however,depending on the observer, when a stereoscopic display switches to aplanar display, there are cases in which it feels natural to theobserver to switch to right-eye data or to left-eye data irrespective ofthe direction in which casing 10 is moved or tilted. In those cases, thedata with which the observer feels natural are display data matching thedominant eye of the observer. Consequently, for the planar display instep 6, it is preferable to provide a function which allows theobserver, on his own, to set data to be sent to the left and right eyepixels.

The second exemplary embodiment is also applicable to the arrangement ofthe third exemplary embodiment.

FIG. 25 is a functional block diagram for such a case, wherein2D.fwdarw.3D return value setting section 93 is added to judging section90. As described above, the arrangement according to the third exemplaryembodiment is different from the second exemplary embodiment in that itdispenses with image swapping control circuit 111 and image generator100 generates different image data. The operation of the third exemplaryembodiment, to which the flowchart shown in FIG. 22 is applicable, isthe same as the operation of the second exemplary embodiment except thatthe image swapping means does not control switching to a planar display(step 16) and the image data applied to the planar display. As describedabove, a function may be provided which allows the image data applied toplanar display to be selected depending on the direction in which casing10 is moved or tilted or to be selected so as to match the dominant eyeof the observer.

The differences between the first and second exemplary embodiments andthe third exemplary embodiment have been described above. The thirdexemplary embodiment does not use an electrooptical device unlike thefirst and second exemplary embodiments. Therefore, though the horizontalresolution for planar display is lower, the manufacturing cost of thedisplay device can be reduced, the casing thereof can be made slightlysmaller, and the reliability of the display device is higher accordingto the third exemplary embodiment.

Fourth Exemplary Embodiment

An arrangement according to a fourth exemplary embodiment resides inusing a display panel which is capable of projecting different images tothe left and right eyes of the observer from a portion corresponding toa pixel which is a minimum display unit that produces an image on astandard planar display panel. The display panel may be, for example, atime-division stereoscopic display panel disclosed in Patent document 1.For example, the display panel may be a display panel having twice morepixels in the horizontal direction along which both eyes of the observerare aligned, than the standard flat display panel.

An example of a display device according to the fourth exemplaryembodiment is illustrated in the front elevational view shown in FIG. 6which has been referred to with respect to the first exemplaryembodiment. As shown in FIG. 7 is a cross-sectional view of casing 10taken along line b of FIG. 6, the display device according to the fourthexemplary embodiment includes display panel 11, image swapping means 13,display controller 12, and operation switch 14 which are housed incasing 10. Display panel 11 comprises a transmissive liquid crystalpanel which includes a plurality of unit pixels, and as the followingfeatures compared with the standard flat display panel:

FIG. 26(a) is a diagram illustrative of a pixel structure of a standardflat display panel. The standard flat display panel comprises a matrixof pixels 4 including six pixels in each horizontal row and three pixelsin each vertical row. Pixels 4 are capable of expressing any desiredgradations, and can express images of 6.times.3 pixels depending oninput data. While each of pixels 4 is of a square shape, the illustratedshape is for illustrative purposes only, and each of pixels 4 may be ofany shape insofar as the ratio of vertical and horizontal dimensionsremains the same when images are expressed of 6.times.3 pixels.

FIG. 26(b) is a diagram illustrative of the pixel structure of thedisplay panel according to the fourth exemplary embodiment, incomparison with FIG. 26(a). The display panel according to the fourthexemplary embodiment comprises a matrix of pixels 41, each made up oftwo sections divided by a vertical line from one of pixels 4 of thestandard flat display panel, including twelve pixels in each horizontalrow and three pixels in each vertical row. Pixels 41 are capable ofexpressing any desired gradations, and can express images of 12.times.3pixels. Since each of pixels 41 has a horizontal dimension which isone-half of its vertical dimension, the size of the display screen isthe same as the size of the display screen of the standard flat displaypanel shown in FIG. 26(a). While each of pixels 41 is of a rectangularshape, the illustrated shape is for illustrative purposes only, and eachof pixels 41 may be of any shape insofar as the ratio of vertical andhorizontal dimensions thereof is 2:1. Image swapping means 13 comprisesa lenticular lens that is arranged as shown in FIG. 27 to cause pixels41 to function horizontally alternately as left-eye pixels 4L andright-eye pixels 4R for stereoscopic display.

The fourth exemplary embodiment may be illustrated in a functional blockdiagram which is the same as FIG. 23 which shows the third exemplaryembodiment.

The fourth exemplary embodiment is different from the third exemplaryembodiment as to image data generated by image generator 100. As withthe previous exemplary embodiment, data to be displayed which are storedin data storage 102 comprise three-dimensional data including depthinformation, and it is preferable to generate two-dimensional image databy arithmetic unit 101 which performs a rendering process on thethree-dimensional data. 3D data used for stereoscopic display, i.e.,two-dimensional image data for the left and right eyes having aparallax, are generated by setting two hypothetical viewpointscorresponding to the left and right eyes of the observer and performinga rendering process.

2D data for planar display, i.e., image data free of a parallax, aregenerated by setting one viewpoint corresponding to the center betweenthe left and right eyes of the observer and performing a renderingprocess. Since the display panel according to the fourth exemplaryembodiment has a resolution which is twice greater in the horizontaldirection, 2D data for planar display may comprise right-eye dataproduced by a rendering process for a stereoscopic display as data forthe left and right eye, or may comprise left-eye data produced by arendering process for a stereoscopic display as data for the left andright eye. 3D data as image data to be generated and 2D data as imagedata to be generated are shown in FIGS. 28(a) through 28(d).

In the fourth exemplary embodiment, data to be displayed which have beensubjected to a rendering process may be stored in data storage 102 inadvance in the format of two-dimensional data corresponding to FIG.28(a) which are free of depth information. The format is widely used foractual content captured using two cameras. Two-dimensional datacorresponding to FIGS. 28(a) and 28(d) may also be stored in datastorage 102. Since these data do not need to be subjected to a renderingprocess, as described above, arithmetic unit 101 and memory 103 may beinexpensive.

In FIG. 27, image swapping means 13 comprises a lenticular lens.However, image swapping means 13 may comprise a parallax barrier. Thoughthe parallax barrier is less bright than the lenticular lens, it allowsthe display device to be manufactured less costly.

The process of determining whether or not both eyes of the observer arepositioned in the stereoscopic viewing area has been described withrespect to the first exemplary embodiment.

The operation of the fourth exemplary embodiment is the same as theoperation of the third exemplary embodiment except for the data forplanar display which are used in step 6 of the flowchart shown in FIG.20.

In the fourth exemplary embodiment, as is the case with the thirdexemplary embodiment, planar display is achieved by sending identicalimage data to right-eye pixels 4R and left-eye pixels 4L. However, theimage data have a different horizontal resolution from the image dataused in the third exemplary embodiment, and may be either image datashown in FIGS. 28(a) through 28(d).

As with the third exemplary embodiment, a function may be provided whichallows the image data applied to planar display to be selected dependingon the direction in which casing 10 is moved or tilted or to be selectedso as to match the dominant eye of the observer.

[Advantages]

The stereoscopic display device according to the fourth exemplaryembodiment has the same resolution for planar display and stereoscopicdisplay since it uses display panel 11 schematically shown in FIG.26(b). When switching to planar display in a flow from step 4 to step 6shown in FIG. 20, the observer does not experience a feeling ofstrangeness whereas in the first and second exemplary embodiments theobserver has a feeling of strangeness due to a change in the horizontalresolution.

In the present exemplary embodiment, the stereoscopic pixel unit whichincludes a single unit pixel comprises a left-eye pixel and a right-eyepixel that are horizontally arrayed as the unit pixel. However, thepresent invention is not limited to such a configuration.

According to another example, while obtaining the same advantages asdescribed above, the present invention can be applied to a process ofprojecting parallax images onto the left and right eyes of the observerbased on light rays emitted in a time-division fashion from unit pixels,with optical distributing means by way of backlight control.

Fifth Exemplary Embodiment

A fifth exemplary embodiment is based on the operation of the secondexemplary embodiment which is applied to the arrangement of the fourthexemplary embodiment described above.

The fifth exemplary embodiment is represented by the functional blockdiagram shown in FIG. 27. The fifth exemplary embodiment is differentfrom the fourth exemplary embodiment in that judging section 90 includes2D.fwdarw.3D return value setting section 93 described above withrespect to the second exemplary embodiment, and will not be described indetail below.

Operation of the fifth exemplary embodiment is essentially the same asthe operation of the second exemplary embodiment. With reference to theflowchart shown in FIG. 22, the fifth exemplary embodiment is differentfrom the second exemplary embodiment as to the image swapping meanswhich is not turned on and off and the image data used in step 16. Inthe fifth exemplary embodiment, image data applied to planar display maybe any items of image data either image data shown in FIGS. 28(a)through 28(d) as described above with respect to the fourth embodiment.In addition, a function may be provided which allows the image dataapplied to planar display to be selected depending on the direction inwhich casing 10 is moved or tilted or to be selected so as to match thedominant eye of the observer.

[Advantages]

Inasmuch as the stereoscopic display device according to the fifthexemplary embodiment has the same resolution for a planar display andstereoscopic display, eliminating the strange feeling that is caused bya change in resolution and also reducing the uncomfortable feeling thatis caused when frequent switching occurs between the stereoscopicdisplay and the planar display is effective.

Sixth Exemplary Embodiment

A sixth exemplary embodiment resides in using a display panel having atleast three viewpoint pixels arranged in a horizontal direction. Thougha unit pixel may be used as each of the viewpoint pixels, the presentexemplary embodiment uses a display panel which includes at least threeviewpoint pixels arranged in a horizontal direction in a portioncorresponding to a pixel which is a minimum display unit that producesan image on a standard planar display panel. Specifically, the presentexemplary embodiment uses a display panel which includes N pixels, whereN represents the number of viewpoints, in a portion corresponding to apixel which is a minimum display unit that produces an image on astandard planar display panel.

The sixth exemplary embodiment where N=4 will be described below.

As shown in FIG. 6, the display device according to the sixth exemplaryembodiment includes display panel 11, image swapping means 13, displaycontroller 12, and operation switch 14 which are housed in casing 10. Asdescribed above, display panel 11 has four pixels in a portioncorresponding to a pixel which is a minimum display unit that producesan image on a standard planar display panel.

FIG. 29 is a diagram illustrative of a pixel structure of the displaypanel used in the sixth exemplary embodiment. The display panelaccording to the sixth exemplary embodiment includes a matrix of pixels,each made up of four sections divided by vertical lines from one ofpixels 4 of the standard flat display panel shown in FIG. 26(a),including twenty-four pixels in each horizontal row and three pixels ineach vertical row. The pixels shown in FIG. 29 are capable of expressingany desired gradations (e.g., on a liquid crystal panel), and canexpress images of 24.times.3 pixels. Since the ratio of a horizontaldimension to a vertical dimension is 1/4, the size of the display screenis the same as the standard flat display panel which is made up of6.times.3 pixels shown in FIG. 26(a). While each of the pixels is of arectangular shape, the illustrated shape is for illustrative purposesonly, and each of the pixels may be of any shape insofar as the ratio ofvertical and horizontal pixel numbers is 4:1. While the display panel ismade of 24.times.3 pixels, the illustrated size is for illustrativepurposes only, and the total number of pixels may be determineddepending on the purpose of the display panel. For a stereoscopicdisplay, the pixels function as first viewpoint pixel 4D, secondviewpoint pixel 4C, third viewpoint pixel 4B, and fourth viewpoint pixel4A which are arranged in a horizontal direction. A lenticular lens asimage swapping means 13 is disposed as shown in FIG. 29.

The stereoscopic display on the display panel according to the sixthexemplary embodiment will be described below.

FIG. 30 is a cross-sectional view of an optical model in which imagesare projected onto observational plane 30 which lies parallel to thesurface of the display panel and is spaced an optimum observationdistance OD therefrom.

The display panel (not shown) comprises a group of a group of lightmodulating elements as a matrix of pixels (e.g., a liquid crystalpanel). FIG. 30 shows only an array of first viewpoint pixel 4D, secondviewpoint pixel 4C, third viewpoint pixel 4B, and fourth viewpoint pixel4A.

Lenticular lens 3 which functions as the image swapping means isdisposed on the front surface (facing observational plane 30) of thedisplay panel. A light source (not shown: so-called backlight) isdisposed on a rear surface (remote from lenticular lens 3) of thedisplay panel. Lenticular lens 3 comprises a linear array of cylindricallenses 3 a each in the form of a one-dimensional lens having ahog-backed convex shape. Lenticular lens 3 does not have a lens effectin its longitudinal direction, but has a lens effect only in its arraydirection which is perpendicular to the longitudinal direction.Lenticular lens 3 is arranged such that its longitudinal direction isperpendicular to the direction along which first viewpoint pixel 4D,second viewpoint pixel 4C, third viewpoint pixel 4B, and fourthviewpoint pixel 4A are arranged. One cylindrical lens 3 a is assigned toeach set of pixels 4D, 4C, 4B, 4A.

Light emitted from each pixel is deflected by lenticular lens 3 andprojected. Of the light emitted from each pixel, light that passesthrough the principal point (vertex) of closest cylindrical lens 3 a isillustrated as a light ray. Then, there are defined area 74D whereimages are projected from all first viewpoint pixels 4D, area 74C whereimages are projected from all second viewpoint pixels 4C, area 74B whereimages are projected from all third viewpoint pixels 4B, and area 74Awhere images are projected from all fourth viewpoint pixels 4A. Thepitch of each pixel is represented by P, and the width of a projectedimage on observational plane 30 which is spaced from the pixels byoptimum observation distance OD is represented by P′.

FIG. 30 is a diagram showing an optical model in which right eye 55R ofobserver 50 is positioned in area 74B and left eye 55L of observer 50 ispositioned in area 74C.

If there is a parallax between image data sent to second viewpointpixels 4C and third viewpoint pixels 4B, then the observer recognizesthe displayed image as a stereoscopic image. The right eye of theobserver may be positioned in area 74A and the left eye of the observermay be positioned in area 74B. According to the sixth exemplaryembodiment, the observer can enjoy various combinations of parallaximages between area 74A and area 74B, as shown in FIG. 31. If an imageto be displayed which is projected between area 74A and area 74B is animage rendered from four viewpoints, then the observer can enjoy astereoscopic image from different angles and at the same time can begiven a motion parallax by changing its observational position,resulting in an increased stereoscopic effect.

The sixth exemplary embodiment is represented by the same functionalblock diagram as the fourth exemplary embodiment, which is shown in FIG.23. Though the sixth exemplary embodiment is represented by the samefunctional block diagram, image generator 100 generates different databecause the sixth exemplary embodiment uses a different display panel.

As with the previous exemplary embodiment, data to be displayed whichare stored in data storage 102 comprise three-dimensional data includingdepth information, and it is preferable to generate two-dimensionalimage data by arithmetic unit 101 which performs a rendering process onthe three-dimensional data. 3D data used for stereoscopic display, i.e.,four two-dimensional image data having a parallax, are generated bysetting four hypothetical viewpoints and performing a rendering process.

2D data for planar display, i.e., image data free of a parallax, aregenerated by setting one viewpoint corresponding to the center betweenthe left and right eyes of the observer and performing a renderingprocess. Since the display panel according to the sixth exemplaryembodiment has a resolution which is four times greater in thehorizontal direction, 2D data for planar display may comprise one of thedata images from among the data (four images) produced by a renderingprocess for stereoscopic display.

3D data as image data to be generated and 2D data as image data to begenerated are shown in FIGS. 32(a) and 32(b).

Data to be displayed which have been subjected to a rendering processmay be stored in the data storage in advance in the format oftwo-dimensional data corresponding to FIG. 32(a) which are free of depthinformation. For example, the format is capable of handling actualcontent captured using four cameras. Two-dimensional data correspondingto FIGS. 32(a) and 32(b) may also be stored in the data storage. Sincethese data do not need to be subjected to a rendering process, thearithmetic unit and the memory may be inexpensive.

As described above, image generator 100 generates 2D/3D data dependingon the signal from judging section 90, and outputs the generated 2D/3Ddata to display panel driving circuit 110.

In the arrangement according to the sixth exemplary embodiment, imageswapping means 13 comprises a lenticular lens. However, image swappingmeans 13 may comprise a parallax barrier. Although the parallax barrieris not as bright as the lenticular lens, it enables a reduction in themanufacturing cost of the display device.

In the sixth exemplary embodiment, the conditions for determiningwhether or not both eyes of the observer are positioned in thestereoscopic viewing area may be based on boundary line information ofdiamond-shaped areas 74A through 74D shown in FIG. 30. Furthermore,combination of areas 74A through 74D shown in FIG. 31 are allowed. Inaddition, if there is little parallax between images projected onto area74A and area 74B, that causes a dual image to be less noticeable, thenthe positions where both images can be seen as shown in FIG. 33 are alsoallowed. According to the present exemplary embodiment, therefore, theconditions for stereoscopic vision should preferably determined to matchthe preference of the observer.

[Description of the Operation]

Operation of the sixth exemplary embodiment can be described withreference to the flowchart shown in FIG. 20 as with the operation of thethird and fourth exemplary embodiments. The operation of the sixthexemplary embodiment is different from the operation of the third andfourth exemplary embodiments only in that data for planar display instep 6 are different. The planar display according to the sixthexemplary embodiment is achieved by equalizing all the image data sentto the first through fourth viewpoint pixels. The image data areillustrated as either the image data items shown in FIG. 32(a) or theimage data shown in FIG. 32(b).

As described above in the other exemplary embodiments, stereoscopicvision may be judged in step 4 shown in FIG. 20 by applying theconditions for judging stereoscopic vision which are derived from thedesigning conditions to the initial settings, allowing the observer tomove and tilt casing 10 to look for limitations on stereoscopic vision,and storing conditions for limiting stereoscopic vision, whileperforming stereoscopic vision in step 1. As described above withrespect to the conditions for judging stereoscopic vision, this processis particularly effective in the sixth exemplary embodiment.

In step S6 for planar display, the first viewpoint data may be sent tothe four types of pixels when casing 10 is moved in the negativedirection along the X-axis or tilted to the left at the time planardisplay is judged, and the fourth viewpoint data may be sent to the fourtypes of pixels when casing 10 is moved in the positive direction alongthe X-axis or tilted to the right at the time planar display is judged.The feeling of strangeness that occurs in an observer when switchingfrom a stereoscopic display to a planar display can be reduced byswitching between the data, as described above.

According to our research, however, depending on the observer, whenstereoscopic display switches to planar display, there are cases whenswitching to the first and second viewpoint data or the third and fourthviewpoint data, irrespective of the direction in which casing 10 ismoved or tilted; feels natural to the observer. In those cases, displaydata that match the dominant eye of the observer are data that feelsnatural to the observer. Consequently, for planar display in step 6, itis preferable to provide a function which allows the observer, on hisown, to set data to be sent to the four types of pixels.

The sixth exemplary embodiment has been described above as being appliedto a display panel having pixels of four viewpoints. However, the numberof viewpoints may be represented by N, and image generator 100 maygenerate image data of N viewpoints.

Seventh Exemplary Embodiment

A seventh exemplary embodiment is based on the operation of the secondexemplary embodiment which is applied to the arrangement of the sixthexemplary embodiment described above, and is different therefrom withregard to the operation after having switched to planar display untilstereoscopic display is performed again.

The arrangement according to the seventh exemplary embodiment is thesame as the sixth embodiment except that judging section 90 includes2D.fwdarw.3D return value setting section 93, and will not be describedin detail below. As with the sixth exemplary embodiment, the displaypanel according to the seventh exemplary embodiment has pixels of fourviewpoints. However, the number of viewpoints may be represented by N,and image data of N viewpoints may be generated.

Operation of the seventh exemplary embodiment can be described withreference to the flowchart shown in FIG. 22 as with the operation of thethird and fifth exemplary embodiments. As with the sixth exemplaryembodiment, the image data for planar display in step 16 are illustratedas either the image data shown in FIG. 32(a) or the image data shown inFIG. 32(b).

As described above with respect to the sixth exemplary embodiment, afunction may be provided which allows the image data applied to planardisplay in step 16 to be selected depending on the direction in whichcasing 10 is moved or tilted or to be selected so as to match thedominant eye of the observer.

With the stereoscopic display device according to the seventh exemplaryembodiment, as with the sixth exemplary embodiment, the observer canenjoy a stereoscopic image from different angles and at the same timecan be given a motion parallax, resulting in an increased stereoscopiceffect.

The present invention is applicable to portable information terminals(terminal devices) such as mobile phones, portable personal computers,portable game machines, portable media player, etc.

As described above, the stereoscopic display device according to thepresent invention detects movement of the casing thereof and projects aparallel-free image in a situation wherein a stereoscopic display is notappropriate, thereby preventing the observer from feeling discomfort andalso preventing the observer from suffering symptoms such as vertigo andmotion sickness. Since the stereoscopic viewing area is judged bydetecting movement of the casing and performing calculations, thedisplay device is less expensive than conventionalline-of-vision-tracking display devices which require a camera, an imageprocessing function for detecting viewpoint positions, and an infraredirradiator.

While the present invention has been described above with respect to theexemplary embodiments, the present invention is not limited to the aboveexemplary embodiments. Various changes that can be understood by thoseskilled in the art can be made to the configuration and details of thepresent invention within the scope of the present invention.

What is claimed is:
 1. A display handled device for displaying an imagein either a stereoscopic display or a planar display, comprising: adetector that detects movement of the display device; a judging sectionthat compares a value representative of the movement of the displaydevice which is detected by said detector with a preset threshold value;an image generator that generates and outputs, from among at least twoimage data items, either image data having a parallax or image data freeof a parallax, based on a result of the comparison made by said judgingsection; a display panel that displays the image data output by saidimage generator, said display panel comprising a plurality of unitpixels; and image swapping means that controls the projection of theimage data displayed by said display panel from said display panel;wherein said detector detects an angle of tilt of the display device asthe movement of the display device; said judging section comprises: afunction for an observer to record the angle of tilt for a desiredposition where said stereoscopic display is possible as an initialvalue; a first judging function to compare the angle of tilt dependingon the movement of said display device with a preset angle-of-tiltthreshold value, and judge said stereoscopic display if the angle oftilt is smaller than the angle-of-tilt threshold value, otherwise judgesaid planar display; and a second judging function to set the angle oftilt for shifting from said planar display to said stereoscopic displayas a 2D-3D return value based on the initial value, compare the angle oftilt depending on the movement of said display device with the set 2D-3Dreturn value at a time of said planar display, and judge saidstereoscopic display if the angle of tilt falls within the 2D-3D returnvalue, or judge said planar display otherwise, and said image generatorgenerates the image data having the parallax if said stereoscopicdisplay is judged by said first or second judging function of saidjudging section, and generates the image data free of the parallax ifsaid planar display is judged by said first or second judging functionof said judging section.
 2. The display device according to claim 1,wherein said detector comprises an acceleration sensor, a geomagneticsensor, a gyrosensor, or an angular velocity sensor.
 3. The displaydevice according to claim 1, wherein said detector further detects adistance of movement of said display device as the movement of saiddisplay device, said judging section comprises: a function to furtherrecord the distance of movement for a desired position where saidstereoscopic display is possible as an initial value; a first judgingfunction to further compare the distance of movement depending on themovement of said display device with a preset distance-of-movementthreshold value, and judge said stereoscopic display if the angle oftilt is smaller than the angle-of-tilt threshold value, and the distanceof movement is smaller than the distance-of-movement threshold value, orjudge said planar display otherwise; and a second judging function tofurther set the distance of movement for shifting from said planardisplay to said stereoscopic display as a 2D-3D return value based onthe initial value, compare the angle of tilt and the distance ofmovement depending on the movement of said display device with therespective set 2D-3D return values at the time of said planar display,and judge said stereoscopic display if the angle of tilt and thedistance of movement fall within the respective 2D-3D return values, orjudge said planar display otherwise, and said image generator generatesthe image data having the parallax if said stereoscopic display isjudged by said first or second judging function of said judging section,or generates the image data free of the parallax if said planar displayis judged by said first or second judging function of said judgingsection.
 4. The display device according to claim 3, wherein saiddetector comprises an acceleration sensor, an ultrasonic sensor, or asmall-size camera.
 5. A display handled device for displaying an imagein either a stereoscopic display or a planar display, comprising: adetector that detects movement of the display device; a judging sectionthat compares a value representative of the movement of the displaydevice which is detected by said detector with a preset threshold value;an image generator that generates and outputs, from among at least twoimage data items, either image data having a parallax or image data freeof a parallax, based on a result of the comparison made by said judgingsection; a display panel that displays the image data output by saidimage generator, said display panel comprising a plurality of unitpixels; and image swapping means for controlling the projection of theimage data displayed by said display panel from said display panel;wherein said judging section comprises: a function for an observer torecord the value representative of the movement for a desired positionwhere said stereoscopic display is possible as an initial value; a firstjudging function to compare the value representative of the movementdepending on the movement of said display device with a preset thresholdvalue, and judge said stereoscopic display if the value representativeof the movement is smaller than the threshold value, otherwise judgesaid planar display; and a second judging function to set a valuerepresentative of movement for shifting from said planar display to saidstereoscopic display as a 2D-3D return value based on the initial value,compare the value representative of the movement depending on themovement of said display device with the set 2D-3D return value at atime of said planar display, and judge said stereoscopic display if thevalue representative of the movement falls within the 2D-3D returnvalue, or judge said planar display otherwise, and said image generatorgenerates the image data having the parallax if said stereoscopicdisplay is judged by said first or second judging function of saidjudging section, or generates the image data free of the parallax, andselects and outputs an arbitrary viewpoint image depending on thedirection of the movement of said display device which is detected bysaid detector if said planar display is judged by said first or secondjudging function of said judging section.
 6. The display deviceaccording to claim 1, wherein said stereoscopic display and said planardisplay have the same resolution.
 7. The display device according toclaim 1, wherein at least two viewpoint images are displayed in saidstereoscopic display.
 8. The display device according to claim 1,wherein said image swapping means comprises an electrooptical device. 9.The display device according to claim 8, wherein said image swappingmeans is turned on if said stereoscopic display is judged by said firstor second judging function of said judging section, and is turned off ifsaid planar display is judged by said first or second judging functionof said judging section.
 10. The display device according to claim 1,wherein said image generator generates the image data having theparallax irrespective of a result of the comparison made by said judgingsection, and said display panel displays at least two image data itemshaving the parallax using said unit pixels if said stereoscopic displayis judged by said first or second judging function of said judgingsection, and displays one image data item of the image data items havingthe parallax using said unit pixels if said planar display is judged bysaid first or second judging function of said judging section.
 11. Thedisplay device according to claim 1, wherein said display panel includesa stereoscopic pixel unit comprising at least two of the unit pixelseach comprising a right-eye pixel and a left-eye pixel, and displayssaid image data using said stereoscopic pixel unit.
 12. The displaydevice according to claim 1, wherein said display panel includes astereoscopic pixel unit comprising one of the unit pixels eachcomprising a right-eye pixel and a left-eye pixel which are arrayed in ahorizontal direction, and displays said image data using saidstereoscopic pixel unit.
 13. The display device according to claim 1,wherein said display panel includes a stereoscopic pixel unit comprisinga horizontal array of viewpoint pixels each comprising at least three ofthe unit pixels, and displays said image data using said stereoscopicpixel unit.
 14. The display device according to claim 1, wherein saiddisplay panel includes a stereoscopic pixel unit comprising a horizontalarray of at least three viewpoint pixels in one of the unit pixels, anddisplays said image data using said stereoscopic pixel unit whichcomprises one of the unit pixels.
 15. The display device according toclaim 1, wherein said image generator develops an amount of parallaxdepending on depth information, with respect to data to be displayedwhich has said depth information.
 16. The display device according toclaim 1, wherein said image generator generates said at least two imagedata having the parallax as two left- and right-eye image data to bedisplayed and generates said image data free of the parallax as centralimage data to be displayed between said two left- and right-eye imagedata.
 17. The display device according to claim 1, wherein said imagegenerator generates said image data free of the parallax by selectingand outputting an arbitrary viewpoint image depending on the dominanteye of the observer which is set from outside of the display deviceirrespective of the direction of the movement of the display devicewhich is detected by said detector.
 18. A display handled device fordisplaying an image in either a stereoscopic display or a planardisplay, comprising: a detector that detects movement of said displaydevice; a judging section that compares a value representative of themovement of said display device which is detected by said detector witha preset threshold value; an image generator that generates and outputs,from among at least two image data items, either image data having aparallax or image data free of a parallax, based on a result of thecomparison made by said judging section; a display panel that displaysthe image data output by said image generator, said display panelcomprising a plurality of unit pixels; and image swapping means thatcontrols a projection of the image data displayed on said display paneloutwardly from said display panel, wherein the image data having theparallax includes at least one right-eye data item and at least oneleft-eye data item, based on a result of the comparison made by thejudging section, said image generator generates the image data havingthe parallax if the value representative of the movement of said displaydevice is smaller than the threshold value, or otherwise outputs theleft-eye data item as the image data free of the parallax if themovement of said display device detected by said detector is a rightdirection of view of an observer, or if the tilt is a rotation such thata left end of said display panel moves toward the observer if viewedfrom the observer, and said image generator outputs the right-eye dataitem as the image data free of the parallax if the movement of saiddisplay device detected by said detector is a left direction of view ofthe observer, or if the tilt is a rotation such that a right end of saiddisplay panel moves toward the observer if viewed from the observer. 19.A terminal device including a display device according to claim
 1. 20. Adisplay method for displaying an image on a display handled device,comprising: detecting of movement of said display device; comparing avalue representative of the detected movement of said display devicewith a preset threshold value; generating and outputting of, from amongat least two image data items, either image data having a parallax orimage data free of a parallax, based on a result of said comparing; anddisplaying of the generated image data, wherein said detecting detectsan angle of tilt and a distance of movement of said display device asthe movement of said display device, and calculates the valuerepresentative of the movement of said display device based on thedetected angle of tilt and distance of movement of said display device,said comparing compares the calculated value representative of themovement of said display device with the threshold value, and based on aresult of said comparing, said generating generates the image datahaving the parallax if the value representative of the movement of saiddisplay device is smaller than the threshold value, or otherwisegenerates the image data free of the parallax, and selects and outputsan arbitrary viewpoint image depending on the value representative ofthe movement of said display device.
 21. A display method for displayingan image in either a stereoscopic display or a planar display on adisplay handled device, comprising: detecting of movement of saiddisplay device; recording of the detected movement as an initial value;setting of a 2D-3D return value for shifting from said planar display tosaid stereoscopic display based on the recorded initial value; firstcomparing the detected movement of said display device with a presetthreshold value; second comparing the detected movement of said displaydevice with the 2D-3D return value; judging on said image display to beeither said stereoscopic display or said planar display based on resultsof said first comparing and said second comparing; generating of, fromamong at least two image data items, either image data having a parallaxor image data free of a parallax based on a result of the judging; anddisplaying of the generated image data; wherein, said detecting detectsan angle of tilt of said display device, said recording records thedetected angle of tilt for a desired position where said stereoscopicdisplay is possible as the initial value, said setting sets an angle oftilt for shifting from said planar display to said stereoscopic displayas the 2D-3D return value based on the initial value, said firstcomparing compares the detected angle of tilt with a presetangle-of-tilt threshold value, based on a result of said firstcomparing, said judging judges said stereoscopic display if the angle oftilt is smaller than the angle-of-tilt threshold value, or judges saidplanar display otherwise, said second comparing compares the detectedangle of tilt with the 2D-3D return value, at a time of said planardisplay, based on a result of said second comparing, said judging judgessaid stereoscopic display if the angle of tilt falls within the 2D-3Dreturn value, or judges said planar display otherwise, and based on aresult of the judging, said generating generates the image data havingthe parallax if said stereoscopic display is judged, or generates theimage data free of the parallax if said planar display is judged. 22.The display method according to claim 21, wherein said detecting furtherdetects a distance of movement of said display device, said recordingfurther records the detected distance of movement for a desired positionwhere said stereoscopic display is possible as the initial value, saidsetting sets an angle of tilt for shifting from said planar display tosaid stereoscopic display as the 2D-3D return value based on the initialvalue, said first comparing further compares the detected distance ofmovement with a preset distance-of-movement threshold value, based on aresult of said first comparing, said judging judges said stereoscopicdisplay if the angle of tilt is smaller than the angle-of-tilt thresholdvalue, and the distance of movement is smaller than thedistance-of-movement threshold value, or judges said planar displayotherwise, said second comparing further compares the detected distanceof movement with the 2D-3D return value, at the time of said planardisplay, based on a result of said second comparing, said judging judgessaid stereoscopic display if the angle of tilt and the distance ofmovement fall within the respective 2D-3D return values, or judges saidplanar display otherwise, and based on a result of said judging, saidgenerating generates the image data having the parallax if saidstereoscopic display is judged, and generates the image data free of theparallax if said planar display is judged.